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FORWARD 

This  effort  had  as  its  purpose  the  study  oft 

(a)  Randomisation  Tests  of  Statistical  Inference.  These  tests 
may  be  utilised  to  minimise  the  risk  of  biased  results  resulting  from 
non-random  sets  of  failure  data.  Sines* ,  generally,  reliability 
estimations  must  be  based  upon  the  assumption  that  the  occurring 
failures  are  random  in  nature,  suoh  tests  of  randomness  may  be  con¬ 
sidered  as  a  necessity. 

(b)  Statistical  Decision  Functions.  These  functions  may  be 
used  to  determine  the  advisability  of  terminating  reliability  tests  in 
the  presence  of  relatively  sparse  data.  From  decision  functions,  risk 
functions  can  be  developed  which  may  be  capable  of  mathematically 
equating  potential  costs  to  each  possible  alternate  decision  (accept, 
reject,  or  continue  test). 

In  the  area  of  randomisation  tests  the  contractor  has  investi¬ 
gated  and  modified  (for  reliability  test  purposes)  two  testing  procedures, 
the  runs  test  and  the  serial  correlation  test. 

Of  the  two,  the  runs  test  is  the  easiest  to  apply  (can  be 
implemented  using  a  simple  table).  The  runs  test  can  also  be  used  to 
determine  whether  or  not  a  modification  or  redesign  of  an  equipment  has 
resulted  in  a  significant  improvement  in  reliability. 

The  serial  correlation  test,  although  more  involved  than  the 
runs  test,  can  readily  be  applied  if  one  is  willing  to  set  up  and  solve 
a  series  of  elementary  statistical  relationships. 

In  the  area  of  statistical  decision  functions  the  oontraotor  has 


iii 


performed  a  literature  searob  and  analysis  of  all  available  data  on 
the  subject.  The  conclusion  was  reached  that  it  nay  well  be  possible 
to  develop  a  risk  function  for  reliability  test  purposes.  However, 
before  this  is  accomplished  more  knowledge  concerning  appropriate 
decision  functions  and  their  practical  ramifications  must  be  acquired. 
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ABSTRACT 

This  is  a  study  of  several  randomisation  tests  of  statistical 
inference.  For  testing  the  randomness  of  a  sequence  of  observations, 
the  Runs  Test  and  the  Serial  Correlation  Test  are  discussed. 

Some  aspects  of  decision  theory  are  discussed.  Among  the 
topics  considered  are  definitions,  basic  principles,  the  pay-off  matrix, 
utility  theory,  and  principles  of  choice.  The  report  concludes  with 
critical  comments  on  utility  theory  and  principles  of  choice. 
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Development  of  Non-Parametric  Techniques 
to  the 

Reliability  Testing  of  Air  Foroe  Ground  Electronic  Equipment 

INTRODUCTION 

Randomisation  tests  are  necessary  to  avoid  biased  results  when 
the  experiment  (as  is  almost  always  the  case)  is  based  on  conditions  of 
random  sampling.  If  there  is  any  slight  suspicion  that  the  observations 
do  not  form  a  random  set  when  taken  over  some  time  interval,  it  is  im¬ 
portant  to  test  for  randomness  before  applying  any  statistical  technique. 
The  Runs  Test  and  the  Serial  Correlation  Test,  which  can  be  used  to 
determine  a  lack  of  randomness  in  sequences  of  observations  if  such  a 
lack  exists,  are  discussed.  These  tests  are  nonparametric  since  they  can 
be  used  when  the  underlying  frequency  is  unknown.  Examples  are  given  which 
illustrate  how  the  Runs  Test  can  be,  used  to  test  for  the  identity  ,of  two 
distributions. 

The  works  of  von  Neumann  and  Morgenstern  (1944,  1947)  and  Wald 
(1950)  concerning  the  theory  of  games  and  the  theory  of  statistical 
decisions  stimulated  much  of  the  basic  research  in  decision  theory  during 
the  last  decade.  In  the  second  part  of  this  report  it  is  our  purpose  to 
discuss  some  of  the  basic  concepts  of  this  theory.  Decision  funotions 
can  often  be  used  to  determine  a  rule  for  carrying  out  the  reliability 
experimentation  and  for  making  a  terminal  decision.  In  this  report,  we 
will  point  out  the  difficulties  encountered  in  bridging  the  gap  between 
theory  and  practice. 


DISCUSSION 
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1.  Randomisation  Teats 

a)  Random  Sampling 

A  set  of  elements  which  have  a  common  measurable  or  observable 
characteristic  is  usually  called  the  population  or  the  universe.  A 
subset  of  these  elements,  chosen  in  any  manner,  is  called  a  sample  of 
the  population. 

One  type  of  population  consists  of  elements  or  observations 
which  actually  exist.  Examples  of  this  type  of  universe  might  be: 

(a)  all  the  registered  voters  of  Philadelphia  and  (b)  the  seven  members 
of  the  Board  of  Education  of  a  certain  city.  In  studying  some 
characteristic  of  the  voters  of  Philadelphia,  it  would  be  impractical 
to  contact  eveiy  voter  whereas  it  would  be  reasonable  to  contact  only 
a  subset  or  sample  of  this  population.  Usually  it  is  possible  to  obtain 
adequate  information  for  most  purposes  from  relatively  small  samples. 

In  the  case  of  the  Board  of  Education  members,  because  of  the  small 
number,  it  probably  would  be  easy  to  measure  every  element  (individual) 
of  the  population.  Both  of  the  above  were  illustrations  of  finite 
populations.  A  second  type  of  population,  usually  involved  in  experi¬ 
ments,  is  obtained  if  we  consider  all  the  hypothetical  measurements  cf 
the  weight  of  an  object.  Also,  the  population  consisting  of  all  the 
hypothetical  tosses  of  a  die  is  a  infinite  population  since  "all 
possible  tosses”  can  never  be  made. 

The  purpose  of  a  statistical  test  is  to  make  some  generalisation 
/the  meantime  between  failures  is  X  hour/7  about  a  population  from  a 
subset  or  sample  of  the  population.  The  way  we  choose  this  sample  plays 


an  important  part  in  the  degree  of  confidence  which  we  can  put  on  the 
results  of  the  experiment.  If  some  individual  elements  of  our  popula¬ 
tion  are  more  likely  to  be  chosen  than  others,  then  the  sample  is  certainly 
biased.  Whether  the  .population  is  finite  or  infinite,  we  would  like  every 
element  of  the  population  to  have  an  equal  chance  of  being  included  in 
the  sample.  A  sample  satisfying  this  condition  is  called  a  random  sample. 
This  definition  implies  that  some  device  must  be  used  so  that  selection 
of  elements  of  the  sample  be  left  to  chance.  However,  in  reliability 
tests  aimed  at  verifying  a  mean  time  to  failure,  after  each  failure,  the 
time  to  failure  is  recorded,  the  equipment  is  repaired  and  then  put  back 
on  test.  The  assumption  is  that  failure  i  and  failure  i+1  are  independent. 
It  is  important  to  test  this  assumption  since  most  techniques  for. testing 
statistical  hypotheses  assume  a  random  set  from  some  population.  In 
particular,  in  the  median  and  other  tests  which  involve  sequences  of  ob¬ 
servations,  it  is  assumed  that  these  sequences  are  random.  Therefore, 
before  applying  these  statistical  methods,  some  methods  must  be  used  to 
test  the  randomness  of  the  sequence.  The  following  two  sections  disouss 
such  techniques. 

b)  The  Run  Test 

One  of  the  most  useful  and  easiest  to  apply  tests  of  the  random¬ 
ness  of  a  sequence  is  the  runs  test.  In  order  to  illustrate  the  use  of 
this  test  oonsider  the  following  set  of  times  to  failure  of  a  certain 
electronic  equipment  gathered  over  a  period  of  several  months: 

(I)  31,  27,  37,  41,  32,  36,  28,  23,  41,  30,  37,  24,  19,  26,  39, 

35,  25,  28,  40,  23. 

We  then  found  the  median.  In  this  oase,  sinoe  we  have  10  elements  above 
and  including  31  and  10  elements  below  and  including  30,  the  median  is 
30.5.  We  now  replace  eaoh  mean  time  between  failures  by  +  if  above  the 
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median  and  by  -  if  below  the  median  of  30.5  This  yields  the  following 
sequence t 

( II )  +-++-H- — +-+ — ++ — +- 

A  run  is  defined  as  a  sequence  of  identical  letters  or  symbols  which 
is  followed  and  preceded  by  different  letters  or  no  symbols  at  all. 

In  the  above  sequence  of  +  and  -  signs,  we  find  that  we  first  have  a 
run  of  1 ,  then  another  run  of  1 ,  and  then  a  run  of  4,  next  a  run  of  2 
etc.  for  a  total  number  of  12  runs.  This  total  number  of  runs  whioh  we 
will  call  ij'is  often  a  good  indioation  of  a  possible  lack  of  randomness. 
For  example,  \f  would  equal  3  if  the  twenty  elements  of  sequence  (II) 
were  arranged  as  follows: 

(III)  WIIU' - ++*■ 

This  could  mean  that  we  have  too  few  runs,  a  total  number  much  smaller 
than  that  expected  under  the  randomness  hypothesis.  In  another  example, 
CT  =  18  if  the  twenty  elements  of  sequence  (II)  were  arranged  in  some 
alternating  or  almost  alternating  pattern  such  as: 

(IV)  +_+_+_+ W++-+-+-+-+ — 

This  probably  means  that  we  have  ton  wnv  runs,  a  number  much  larger 
than  we  could  expeot  by  chanoe.  In  either  of  the  last  two  illustrations 
we  probably  would  reject  the  hypothesis  of  randomness. 

In  order  to  determine  whether  yf',  the  observed  total  number  of 
runs,  is  too  few  or  too  many,  let  us  consider  an  arrangement  of  n^  letters 
or  symbols  of  one  kind  and  letters  or  symbols  of  the  second  kind.  If 
we  assume  that  this  sequence  is  a  random  sample  from  a  given  population, 
it  is  possible  to  obtain  the  sampling  distribution  of  the  variable  tr 
for  repeated  random  arrangements  by  the  laws  of  probability.  These 
probabilities  have  been  used  to  construct  tables  which  enable  us  to 
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test  whether  a  sample  value  of  if  is  unusually  small  or  large.  The  following 
are  portions  of  tables  complied  for  several  levels  of  signifieanoe  which 
enable  us  to  make  exact  tests  when  n^  and  are  small  (less  than  20) t 


TABLE  I— Critical  values  of in  the  Runs  Test  for  ^^5  azy^ 
<Ref*  6) 


“2 

n1 

5 

8 

10 

12 

14 

16 

18 

20 

3 

2 

7 

2 

7 

2 

7 

2 

7 

3 

7 

3 

7 

3 

7 

5 

2 

9 

3 

10 

3 

11 

4 

11 

4 

11 

4 

11 

5 

11 

5 

11 

8 

4 

13 

5 

14 

6 

15 

6 

15 

6 

16 

7 

16 

7 

16 

10 

6 

15 

7 

16 

7 

17 

8 

18 

8 

18 

9 

19 

12 

7 

18 

8 

19 

9 

20 

0 

<v 

O' 

10 

21 

14 

9 

20 

10 

21 

10 

22 

11 

23 

16 

11 

22 

11 

24 

12 

24 

18 

12 

25 

13 

26 

20 

14 

27 

The  values  listed  are  such  that  a  number  less  than  or  equal  to  the 
u^025  v*1'w  (in  the  upper  left  hand  corner  of  each  rectangle)  will  occur  not 
more  than  2 %l  of  the  time;  and  a  number  greater  than  or  equal  to  iT'q~c 
value  (in  the  lover  right  hand  comer  of  rectangle )  will  occur  not  more 
than  2.5$  of  the  time. 


Suppose  we  have  the  following  set  of  times  between  failure 
of  a  certain  electronic  equipment!  31,  27,  37,  41,  32,  36,  23,  23, 
41,  30,  37,  24,  19,  26,  39,  35,  25  ,  28  ,  40,  23.  We  wish  to  test  the 
randomness  of  this  sequence  of  observations  at  the  5%  level  of  sig¬ 
nificance  . 


Solution- 

As  shown  on  page  3,  the  median  of  this  sequence  is  30.5. 
Designating  a  mean  time  between  failures  above  the  median  by  +  and  mean 
time  between  failures  below  the  median  by  -,  the  above  sequence  becomes: 


The  total  number  of  runs  in  this  sequence  is  12.  The  hypothesis 

H  that  we  wish  to  test  is  that  -Ms  and  -'s  occur  in  random  order.  The 
o 

alternate  hypothesis  H.  is  that  the  order  of  +'s  and  -'s  (or  the  total 
number  of  runs)  is  not  random.  We  use  a  two  tail  test  since  we  cannot 
predict  the  direction  of  the  deviation  from  randomness.  In  this  case, 
n^n^lO  and  the  IT  =12.  From  Table  I,  we  see  that  the  number  12  lies 
between  ^025=6  and  ^"".975=16  and  therefore  we  do  not  reject  the  hypothesis 
of  randomness  at  the  5%  level  of  significance. 

Table  I  can  be  used  when  n^  and  n^  are  equal  to  or  less  than 
20.  More  extensive  tables  are  available  but  i has  been  shown  that  if 
n1  and  n 2  are  greater  than  10,  then  the  sampling  distribution  of.  V*  is 
approximately  normal  with  the  mean  and  standard  deviation  given  by  the 
following: 


(A) 


“l*"2 


+  1 


2n.n  (2n,i4)-n1-n2) 


(B) 
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flMBlft  (2) 

We  wish  to  test  the  randormess  of  a  sequence  of  59  observations 
if  n^=25  and  n2=25  and  the  number  of  runs  is  17,  at  the  level  of 
significance. 


SgJaUgn 


Substituting  in  formulas  (A)  and  (8)  give: 

jut  tr  _  l2ll2SK2iI  +  1  =  26 

50 


(G)  The  formula 

<r  v 

tells  us  the  number  of  standard  deviations  a  particular  Z  is  from  the  mean. 
Substituting  in  this  formula  (C)  we  obtain 

Z  =  12=26  =  =2_  =  .2  6 

3.5  3.5 


Since,  in  a  normal  distribution,  955&  of  the  cases  lie  between  -1.96$- 
and  +1 .96<T  from  the  mean,  we  reject  the  hypothesis  of  randomness.  In 
particular  26^  (1.96)  (3. 5)  or  26^  7  is  the  region  of  acceptance  of  the 
hypothesis  of  randonmess  at  the  55t  level. 

The  runs  test  can  also  be  used  to  determine  whether  or  not  two 
random  samples  are  from  populations  having  the  same  frequency  distribu¬ 
tions.  This  is  useful  in  testing  whether  or  not  a  modification  or  a  re¬ 
design  of  equipment  has  resulted  in  a  significant  improvement.  The 
following  examples  will  illustrate  this  technique. 


Suppose  8  observations  of  the  times  to  failure  of  an  equipment 
have  been  recorded  (sample  A)  and  after  a  redesign,  10  new  measurements 
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•re  made  (sample  B)< 

(A)  23,  32,  51,  43,  45,  33,  58,  37 

(B)  38,  27,  47,  38,  40,  55,  61,  49,  43,  52 

Is  this  apparent  improvement  ffi.  mean  of  40.3  in  A  to  a  mean  of 
45  in  £?  a  significant  one  at  2$£  level? 

SglttUOP 

We  first  order  the  18  observations  of  samples  A  and  B  into  a 
single  sequence  according  to  siae.  We  order  by  a  random  device  those 
observations  which  occur  more  than  once.  fka.  alternative  method  is  to 
break  the  ties  in  all  possible  ways  and  note  the  resulting  value  of  CT*. 

If  all  these  values  of  produce  significant  results  or  all  values  of  'S' 
produce  results  which  are  not  significant,  then  the  ties  present  no 
difficulty^?  We  underline  the  observations  from  Sample  A  to  preserve 
their  identity  when  combined  with  the  observations  of  sample  B: 

21,  27,  32,  21,  2L,  38,  38,  40,  A3,  43,  45,  47,  49,  51,  52, 

55,  52,  61 

This  arrangement  has  12  runs.  The  hypothesis  being  tested 
is  that  the  two  samples  A  and  B  have  the  same  distribution  (that  is, 
the  redesign  has  resulted  in  no  improvement).  A  significant  improve¬ 
ment  would  result  in  very  few  runs.  This  is  because  in  the  single 
ordered  sequence,  a  definite  improvement  would  mean  that  the  majority 
of  the  measurements  in  sample  B  would  fall  in  the  right  portion  of' 
the  sequence  and  thus  reduce  the  number  of  runs.  This  indicates  the 
use  of  a  one-tailed  test.  We  can  use  Table  I  at  the  .025  level.  We 
see  that,  for  n^=8  and  h:;,=10  and  tT-12,  from  this  table  that  12^5  and 
therefore  we  do  not  reject  the  hypothesis.  The  hypothesis  would  be 
rejected  (indicating  a  definite  improvement)  only  if  were  less  than 
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or  equal  to  5. 

JSauaflfl  (4) 

Seise  as  example  (3)  with  samples  A  and  B  as  follows: 

(A)  12,  25,  43,  19,  24,  46,  18,  44 

(B)  29,  11,  49,  56,  61,  36,  58,  53,  38,  72,  53,  64 

Solution 

The  combined  sequence  ordered  according  to  sise  is:  11,  1Z, 

!£,  12,  24,  25,  29,  36,  38,  43,  M,  46,  49  ,  53,  55,  56,  58  ,  61,  64,  72 

Here  n^=8  and  n^=12,  and  i/=5.  from  Table  1  again,  we  note 
that  5 <  6  and  therefore  we  reject  the  hypothesis  of  no  improvement. 

c)  Serial  Correlation  Test 

A  second  test  which  is  useful  for  testing  the  randomness  of  a 
sequence  is  the  serial  correlation  test.  If  we  are  studying  a  sequence 
of  observations  which  are  truly  random,  we  would  not  expect  a  relationship 
between  two  consecutive  elements.  The  probability  that  the  smallest  and 
largest  elements  will  be  consecutive  is  the  same  as  the  probability  that 
the  two  largest  or  two  smallest— -or,  in  fact,  any  two  elements  will  be 
consecutive.  Therefore,  if  we  pair  each  element  with  its  successor  and 
ordinary  techniques  of  correlation  are  used,  we  would  expect  that  perfect 
randomness  would  result  in  a  correlation  of  sero. 

If  our  sequence  consists  of  the  n  elements  X^,  X^,  X^, ....X^,...XQ, 
then  the  successor  of  X^  is  Xg,  of  X^  is  ly  and  X^  is  X^  for  i=1,2,3, ...» 
n-1  and  we  also  define  that  the  successor  of  XQ  Is  X^ .  The  pairings  of 
the  two  variables  X  and  T  which  we  are  to  study  are  as  follows: 

X:  Xg  ....  ....  XQ 

T*  T2  T3  *1+1  ••**  *1 
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If  the  correlation  coefficient  is  close  to  +1  or  to  -1  we  probably  would 
reject  the  hypothesis  of  randomness  (since  +1  represents  perfect  positive 
correlation  and  -1  represents  perfect  negative  correlation)  and  if  the 
correlation  is  sero  or  near  sero,  we  probably  would  accept  the  hypothesis 
that  the  sequence  is  random  since  this  would  indicate  no  or  almost  no 
relation  between  X^  and  It  is  possible  to  obtain  a  sampling  dis¬ 

tribution  of  the  serial  correlation  coefficient  by  the  laws  of  probability 

s  .  X  .  •. 

Even  though  the  formula  for  this  correlation  coefficient  is: 


n 

x-i  X4*i  _n*y 

'T-  *•»  -‘dft 


X3, 

it  is  necessary  only  to  consider: 
n 


CP) 


R  =  X.  X, . .  since  X,  I,  S  and  S  are  unchanged 
1—1  i  it  *  x  y 


the 


various  permutations.  It  has  been  shown  that  ah  ^pproxlinatioh  bfi  the 

distribution  of  R  is  normal  with  a  main  and  standard  deviation  ,as  follows : 

g2  —S'' 

’fa)  JA.  R  =  and; 


+  {£-23,  i. 


(G)  Where  ^  =  X*  +  «*+...  X* 

(H)  The  formula  Z  = 
deviations  a  particular  serial  correlation  ooeffioient  is  from  the  mean 
and  therefore  will  enable  us  to  reject  or  aoeept  the  hypothesis  at  a 
given  level  of  confidence .  As  n  becomes  larger  the  computations  involved 
become  laborious.  However,  modifications,  as  will  be  illustrated  in 


R  - 

“R 


gives  us  the  number  of  standard 
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example  (6)  below,  often  will  simplify  these  computations. 

Using  the  serial  correlation  ooefficient  method,  test  the 
following  sequence  of  12  members  for  randomness * 

5,  17,  18,  8,  19,13,  9,  1,  3,  16,  11,  2  at  the  5*  level  of  significance. 


Solution 

The  given  sequence  produces  the  following  pairings: 

xs  5  17  18  8  19  13  9  1  3  16  11  2 

y:  17  18  8  19  13  9  1  3  16  11  2  5 

n 

We  first  find  H  =  ^  *1  .  *i+1  =  (5)  (17)  +  (17)  (18)  +  (18)  (8)  +  ....  + 

irl 

(2)  (5)  =  1319 

Then,  using  formula  (G)  we  find  ,  S^,  S^,  and  S^s 

51  =  5*17+18+ - +2=122 

52  =  (5)2+(l7)2+(18)2+ - +(2)^=1704 

53  =  (5)3+(17)3+(l8)3+....+(2)3=26,630 

54  =  (5)4+07)4+(l8)4+....+(2)4*438,936 

Now  (E)  gives  >  =  ^^^^^1198  and  (P)  produces: 


and  finally: 


Z  =  = 

fR 


=  131 


1319.Tj,ia 


131 


At  the  5?  level  of  signifioanoe,  a  Z  .65  is  required  to  reject 
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the  hypothesis  and  therefore  the  hypothesis  of  randomness  is  accepted. 

ffimmlfl  (f) 

Again,  using  the  serial  correlation  technique,  we  wish  to  test, 
at  the  5%  level  of  significance,  the  following  sequence  for  randoaness: 
6.74,  6.72,  6.77,  6.79,  6.75,  6.80,  6.82,  6.88,  6.92,  6.89 


SfiMiSB 


Since  the  result  of  the  test  is  not  influenced  by  adding  the  same 
constant  to  each  tern  of  the  sequence  or  by  multiplying  each  term  of  the 
Sequence  by  the  same  constant,  we  add  -6.80  and  then  multiply  edcfr  term 
.of  the  sequence  by  100  to  produce  to  following  simpler  sequences  -6,  -8, 
-3,  -1,  -5,  0,  +2,  +8,  +12,  +9.  The  solution  now  follows  as  in.  the 
preceding  example  ;'V 

Is  -6  -8  -3  -1  -5  +0  +2  +8  +12  +9 

is  +9  -6  -8  -3  -1  -5  0  +2  +8  +12 

R=  (-6)(+9)+(-8)(-6)+(-3)(-8)+....+(+9)(l2)  =  246 

51  —  —6  -8  —3  — 1+.....+9  s  8 

52  =  +  (_8)2  +  ("3)2  +  •••  9)2  =  428 

S3=  (-6) 3  +  (-8)3  +  (-3)'  :•  ...  +(  9)3  =  2096 

S4  =  (-6)4  ^+  (-8)4  +  (-4)4  +  ..  +(  9)4  =  37,508 

/*R=  =  .40.4 

<r«=  /JUjSZi  „  ItLm  _  (40.t)2  = 

«"  R  =  ^16186  +  969  -  1,632  = 

*R=  ^15,523  =  125 
again,  finally 


7  -  SM6+A0.A  _  286. A 

6  ”  125  ■  125 


=  2.3 
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Therefore,  elnoe  the  probability  of  obtaining  an  R  of  this  else  or 
greater  is  approximately  .011,  we  reject  the  hypothesis  of  randomness. 

When  we  correlate  each  member  with  its  successor  as  in  the  above 
problems,  this  is  known  as  serial  correlation  with  lag  1.  The  test  is 
applied  in  the  usual  manner  if  any  other  lag  k  is  used. 

Sample  (?) 

Solve  problem  (5)  using  the  runs  technique 

Solution 

In  this  problem  the  median  is  10.  Using  a  +  sign  to  indicate  an 
element  above  the  median  and  -  sign  to  indicate  an  element  below,  the 
sequence  becomes: 

and  IT,  the  number  of  runs,  is  7.  Using  Table  I  with  n1  =  n^  ~  6,  we  see 
that  3<7<10  and  therefore  we  accept  the  hypothesis  of  randomness  at  5%  level. 

Sample  .($). 

Solve  problem  (6)  using  the  runs  technique. 

SPIGOT 

It  is  obvious  that  here  the  first  5  terms  are  above  the  median  and 
the  last  5  terms  are  below  the  median.  Thus  the  sequence  becomes: 

— — -t  H  +  t  and  l/”  =  2 

Prom  Table  I,  we  see  that  (T  =  2  does  not  fall  between  2  and  9. 

Therefore  we  rejeot  the  hypothesis  of  randomness. 

It  is  clear  from  the  last  two  examples  that  the  computations  involved 
in  the  serial  correlation  test  are  much  more  involved  than  those  for  the 
total  runs  test.  For  large  n,  the  condition  of  normality  is  reasonable, 
and  then  the  serial  correlation  test  nay  be  more  reliable. 


2.  Some  Statistical  Decision  Theory 

a)  Definitions  and  Baaie  Principles. 

A  statistician,  a  technician,  or  any  experimenter  who  makes 
observations  and  measurements  is  finally  faced  with  the  problem  of  making 
a  decision  on  the  basis  of  data  available  (usually  incomplete).  Be  may 
have  to  decide  between  the  acceptance  or  rejection  of  a  given  hypothesis. 

In  sequential  testing,  there  are  three  possible  decisions  he  might  make, 
namely:  accept,  rejeot,  or  continue  testing.  He  may  wish  to  estimate 
the  mean  life  to  failure  of  a  piece  of  equipment  on  the  basis  of  the 
results  of  an  experiment.  Or  in  other  words,  he  must  make  a  decision  as 
to  which  element  Xq  of  a  class  X  should  be  adopted  as  the  estimate  of  the 
true  (but  unknown)  value  of  the  parameter.  These  three  familiar  statistical 
problems  can  be  considered  as  special  cases  of  the  general  decision  problem. 

Principles  must  be  applied  to  aid  in  making  the  decision.  In 
any  particular  problem,  the  principle  which  will  guide  the  decision 
making  will  depend  on  the  nature  of  the  problem  and  often  will  involve 
numerical  constants  which  can  only  be  estimated.  A.  Wald  (Ref.  K) 
developed  the  minimax  principle,  that  is  the  principle  of  selecting  a 
rule  which  minimises  the  maximum  risk  which  could  occur.  This  means 
that  the  investigator  anticipates  the  worst  and  acts  accordingly.  This 
is  frequently  a  vise  action  but  has  been  criticised  as  being  too 
pessimistic,  and  several  useful  modifications  of  the  minimax  theory 
have  been  presented.  A  second  method  originated  by  Bayes  and  used  by 
LaPlace  is  to  assume  that  the  a  priori  probabilities  are  all  equal, 
unless  evidence  to  the  contrary  is  available.  According  to  the  Bayes 
principle,  we  choose  a  decision  function  (defined  later)  so  as  to 
maxi maze  the  average  gain.  A  further  discussion  of  the  various  principles 
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of  choice  will  be  found  in  Motion  d. 

i 

Statistical  decision  theory  and  the  theory  of  gases  are  closely 
related.  Many  statistical  problems  can  be  regarded  as  a  two  person  game 
in  which  the  statistician  plays  against  Nature.  As  an  aid  in  making  a 
decision,  the  statistician  performs  an  experiment  whose  outcome  is  S^(i=1*2,...n) 
The  set  of  all  such  possible  outcomes  will  be  called  S,  the  sample  space. 

The  set  of  all  possible  actions  (or  strategies)  we  call  A.  A  may  consist 
of  only  two  elements  a^ ,  and  a^  as  in  the  ease  where  the  decision  is  to 
accept  or  reject  a  hypothesis.  A  may  contain  three  elements  a^ , 
as  in  sequential  testing  when  the  decision  is  to  accept,  reject  or  continue 
testing.  Also  A  may  contain  an  infinite  number  of  elements  as  in  the  case 
of  point  or  interval  estimation.  A  statistical  decision  function,  d(s)aa 
•'is'. a  rule  associating  a  possible  motion,  a,  with  each  possible  Outcome  S. 


For  example,  we  wish  to  estimate  the  unknown  mean  life JU  of  a  population 
using  a  random  sample  of  sise  n  from  the  population..  We  can  consider 
that  a  possible  action  is  a  statement  which  says  that  the  mean  the 
population  is  the  mean  X  =.~  Xi  where  one  outcome  is  S  =  (X£ .  X^  ...  XQ). 

Here  d(S)  =.  X.  A  second  example  would  be  the  test  of  a  hypothesis  Hq:  jA~  40 
iyhloh  requires  the  rejection  of  the  hypothesis  if  X^45  and  the.  ab^eptance  of 

.  .  V  A  ip.  *\  • 

the  hypothesis  if  1^45.  Here  we  have  two  elements  in  A  namely^|i|^  the 

•  ;  ‘  ,  ..  '  Hi)-'' 

rejection  of  the  hypothesis  and  a^  the  acceptance  of  the  hypothesis.  (This 

is  an  arbitrary  rule  vhlchcould  be  modified  readily  to  include  a  3rd  action, 

continue  testing.)  Here  d(s)  =  a^  if  a  -  (X1,  X^»  . . . ^ XQ)  and  X  £ 

_  ,  n  1=* 

and  d(a)sa2  if  S  *  (X, ,  X^,  ....  ,  X)  and  X  Xj>  45 

i=l 

Considering  the  above  two  examples  as  a  gams,  statistician  versus  nature,  the 
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decisions  arrived  at  are  often  oalled  par*  decisions  since  the  strategies 
or  actions  taken  are  clear-out,  that  is  for  a  given  outcome  a  definite 
action  is  prescribed.  In  other  games,  the  decision  may  depend  on  some 
randomising  device  to  select  an  action  (decision)  from  an  existing 
probability  distribution  over  a  set  of  possible  decisions.  Such  decisions 
are  called  mixed  decisions.  In  the  game  of  Parcbesi,  the  decisions  are 
of  the  mixed  variety  since  the  decision  as  t>  which  marker  to  move 
depends  on  the  roll  of  a  pair  of  dice. 

b)  Utility 

The  major  problem  of  the  statistician  is  how  to  choose  between 
alternate  decision  functions.  Associated  with  each  end  result  facing 
the  decision  maker  is  a  numerically  measured  value  called  utility  and 
in  general  he  should  make  those  decisions  for  which  bis  end  results  would 
have  as  large  a  utility  as  possible.  This  criterion  nay  not  necessarily 
always  be  measured  in  dollars  but  other  valuable  considerations  such 
as  time,  reputation  and  even  life  Itself  may  be  applied. 

Let  us  suppose  the  existence  of  a  parameter  space jft  consisting 
of  elements 8  i=/;Zr3,...>)For  each  there  is  a  corresponding 
probability  distribution  on  S,  the  sample  space.  For  a  fixed  <3  ,  a 
certain  action  a  /jap d(si7  will  yield  an  end  result  i9-)  Now  let 

E  devote  the  set  of  all  possible  end  results  e.  Then  d and  d  together 
determine  a  probability  distribution  on  E.  If  the  experimenter  can 
determine  which  of  two  probability  distributions  he  prefers,  then  a 
utility  can  be  assigned  to  each  of  these.  Therefore  each and  d  will 
have  as  utility,  the  utility  of  the  probability  distribution  which  they 
determine.  A  second  way  of  assigning  a  utility  to  a  pair,  (d ,cO)  is 
possible  if  a  precise  numerical  loss  L  resulting  from  a  given  notion 
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can  be  determined.  This  expected  lose  Is  often  called  the  risk,  regret 
or  inutility. 

The  preceding  utility  theory  is  obviously  very  general  in 
character  and  it  is  often  extremely  difficult  to  determine  a  utility  in 
these  ways. 

A  simple  illustration  of  how  numbers  may  be  assigned  to  basic 
alternatives  is  given  by  Savage.  He  proposes  the  following  problem. 

On  a  given  occasion  a  person  faces  the  decision  of  whether  or  not  to 
carry  an  umbrella.  For  simplicity,  only  two  possible  states  of  nature, 
future  rain  and  future  shine  are  considered.  The  possible  consequences 
are  given  by  this  table: 


^'"^'v^Stato 

Act 

Rain 

Shine 

inconvenience  and 

inconvenience 

wet  feet 

and  slight 

Carry 

embarrassment 

Don't 

Miserable 

bliss 

Carry 

drenching 

unalloyed 

In  other  situations,  the  possible  consequences  of  each  act, 
in  each  state,  could  be  measured  in  dollars  and  in  the  preceding  rain- 
shine  example,  the  following  "income  matrix"  might  be  reasonable: 
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e)  The  Pay-off  Matrix 

The  two  person  gaae  mentioned  in  the  preceding  section  can  be  con¬ 
sidered  as  the  product  of  space  X,  all  possible  strategies  of  player  I, 
and  of  space  I,  all  strategies  of  player  II.  We  define  R(xty)  as  a 
function  which  gives  the  outcome  r  for  each  pair  (x,y),  x  belonging  to  X 
and  y  belonging  to  T.  In  the  theory  of  utility  it  has  been  shown  that 
with  each  r  of  our  outoome  space  R  there  may  be  associated  an  M  (x,y) 
called  the  pay-off  function  which  may  be  regarded  as  the  monetary  gain  or 
"utility"  to  player  II  (the  decision  maker) .  In  the  theory  of  games  there 
Is  a  special  class  of  games  called  two-person  aero  sum  games.  This  is  a 
game  in  which  the  total  payfiff  to  the  two  players  is  Zero  so  that  If  the 
pay-off  to  player  II  is  M(x,y)  then  the  payoff  to  player  I  is  -M(x,y) . 

Our  game  can  be  considered  as  a  matrix  (Mij)  in  which  the  columns 
are  the  strategies  of  the  first  player  and  the  rows  the  strategies  of 
the  second  player: 

I 


II 


X1 

*2 

X3 

....  X.. 

n 

r 

*11 

V 

H31 

*  •  •  *  *m1 

*12 

*22 

"* 

•  •  •  •  *m2 

*13 

*23 

M33 

’  *  '  *  *m3 
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Independently,  the  first  player  selects  the  ith  column  sad  et 
the  sene  tine  the  second  player  selects  the  jth  row.  The  pay-off  to 
the  seoond  player  is  the  entry  of  the  matrix.  The  final  choice  of 
a  decision  function  must  be  based  on  some  reasonable  principle  of  choice. 
Therefore,  before  illustrating  the  pay-off  matrix,  principles  of  choice 
will  be  discussed  somewhat  further  in  the  next  section. 

d)  Principles  of  Choice. 

In  section  (a)  reference  was  Bade  to  several  principles  whioh 
could  aid  the  statistician  in  making  a  decision.  Suppose  for  a  given 
problem  we  have  been  successful  in  assigning  the  utility  U(d;<*>)  to  each 
decision  function  d  for  eacho)  in_/\_.  Then,  as  previously  mentioned, 
the  most  conservative  principle  of  choice  is  the  minimax  criterion.  Here 
we  choose  the  decision  function  so  that  the  min  U(d;£>)  is  the  greatest. 
We  will  illustrate  this  principle  by  devising  a  pay-off  matrix  as  ex¬ 
plained  in  the  preceding  section. 


*1 

*2 

X3 

X4 

h 

5 

0 

4 

5 

H 

0 

3 

6 

3 

y3 

2 

3 

3 

2 

y4 

4 

1 

4 

3 

The  mlnimax  principle  states  that  we  should  choose  the  row  for 
which  the  least  possible  gain  is  a  maximum.  In  the  four  rows  in  the 
pay-off  matrix  above,  the  minimum  gains  are  0,0, 2,1.  Therefore  we 
choose  strategy  y^  since  regardless  of  what  happens  our  minimum  gain 


will  be  2  units. 
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The  second  method  mentioned  in  section  (e)  was  the  Bayes  principle 
or  as  it  is  often  oalled  "the  principle  of  insufficient  reason;"  This 
criterion  asserts  that  if  one  is  "completely  ignorant"  of  the  status  of 
nature,  we  assume  that  the  probabilities  of  all  are  equal  and  we  choose  a 
strategy  which  maximises  the  average  gain.  In  the  pay-off  matrix  above, 
the  average  gain  (utility)  for  the  four  rows  are  ^  and  .  There¬ 

fore  we  choose  y1 . 

The' minimax  principle  of  Wald  as  previously  noted  is  unduly 

pessimistic  since  concentration  is  made  on  the  worst  possible  consequence 

of  each  act.  A  number  of  modifications  have  been  devised  to  remove  this 

objection  and  Hurwls  in  1951  suggested  the  selection  of  a  number 

■cf\  (0  which  he  called  the  pessimism-optimism  index.  For  each  act 

,  _  .  •  ,  •  ■'/ 

(or  row)  of  our  matrix  let  be  the  maximum  utility  number  and ;.mg  the 

smallest  utility  number.  We  now  ohoose  the  row  for  which  o(M^+(i^c<)  m^ 

is  a  maximum.  When<^>=o,  this  is  equivalent  to  Wald's  minimax  principle. 

In  the  previous  example  the  four  rows  give  5o(,  6o<,  3<i+2(l-o0  =  «* +2, 

4o4+l(l-«0  =  3«^>+1.  Koro^=i  the  4  rows  would  give  the  values  ^ 

and  would  be  the  strategy  chosen.  For  o4=  the  results  become 

2  ant*  2  80  /2  is  preferred.  And  finally,  forC^=3/4»  We  have 

4^'  4^’  4^  4^*  80  ^2  preferred.  It  seems  clear  that  for 

any  f2  i8  Pre*’eprs4* 

Several  other  modifications  involve  the  use  of  inutility,  risk 
or  regret.  Savage  suggested  that  ve  minimise  the  maximum  regret  where 
the  regret  is  defined  as  the  amount  that  must  be  added  to  the  utility  to 
equal  the  maximum  utility  pay-off.  Or  it  can  be  also  described  as  the 
difference  between  the  pay-off  for  his  actual  strategy  and  the  pay-off  he 
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would  receive  if  he  knew  in  edveaoe  the  state  of  nature  (Player  I  strategy) . 
Symbolically^  4=mjx  Again,  using  the  previous  natrix  as  an 

example,  the  regret  matrix  associated  with  it  is: 


*1 

*2 

X3 

X4 

*1 

0 

3 

2 

0 

*2 

5 

0 

0 

2 

y3 

3 

D 

3 

3 

y4 

1 

2 

2 

2 

Here  we  have  maximum  regrets  for  the  four  rove  3,  5,  3,  and  2. 
Therefore  we  choose  the  set  which  minimises  the  maximum  risk  (regret) 
which  is  y^. 

The  above  example  illustrates  that  the  use  of  four  different 
principles  of  choice  lead  to  four  different  decisions  regarding  the  choice 
of  strategy.  The  minimax  principle  indicates  the  use  of  strategy  y^;  the 
Bayes  principle  lead*  to  the  choice  of  y^ ;  the  modification  developed  by 
Hurwicz  suggests  the  choioe  of  y^  as  the  preferred  strategy;  and  finally 
the  regret  principle  indicates  the  use  of  strategy  y^. 


e)  Bayes  Estimates,  and  Sequential  Decision  Problems 
In  order  to  estimate  a  parameter tD,  the  statistician  performs  an 
experiment  with  outcome s^S^  where  S  is,  as  previously  stated,  the  set  of 
all  possible  outcomes  under  consideration.  An  event  a  subset  of  S  will! 
have  a  probability  depending  on  cD  i 


<s„> 


(.) 


s6s_ 


f. 
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Then  the  decision  function  d(S)=a  enables  us  to  select  an  action  a  from  the 
set  A  of  all  possible  actions.  If  we  now  assume  that  there  exists  a  numeri¬ 
cal  loss  function  L(d,  w),  then  the  expected  value  of  this  loss  will  be 


defined  toy* 


M(d.O» 


-L 


L  (d,<0)  Po3(S) 


where  d=d(S)  and  the  statistician  obviously  tries  to  minimise  this  expected 
loss.  An  estimate  which  does  minimise  the  expected  loss  is  called  a  Bayes 


estimate . 


In  the  case  of  the  binomial  distribution  where: 

©  s  the  probability  of  success  based  on  the  observation  that  the 
n  trials,  and 
p(x)  =  flcx  .  6*0-8)- 

A(0)  d@  =  probability  that  0  lies  in  the  interval  ©  to  .©Hi© 
f^(x)  =  our  estimate  of  the  parameter  © 

KE  ^fn(x)-0  =  K  ^  jfn(x)-©^  Pn(x)  S  a  constant 


(1  =  expectation  of  loss 
it  is  shown  (Ref  l)  that: 


\  (e)  do 


The  6-  that  minimises  L  in  equation  M  is  the  Bayes  estimate.; 

A  well  known  testing  procedure  is  Wald's  sequential  probability 
ratio  test.  After  each  observation  the  statistician  must  make  a  decision 
whether  to  continue  to  make  observations  or  to  take  final  action  (usually 
accepting  or  rejecting  a  hypothesis) .  Let  us  suppose  that  we  have  to 
decide  between  two  hypotheses  and  of  which  the  a  priori  probabilities 
are  g  and  1-g  respectively.  We  assume  that  there  is  no  loss  in  accepting 
the  true  hypothesis,  and  a  loss  if  we  aooept  Hg  when  is  true,  and 
a  loss  of  >P21  if  we  aooept  &j  when  B2  is  true.  If  g  is  small  we  of  course 
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accept  ilnoa  the  risk  R,=  jf^g  la  small.  Also  if  g  ia  large,  1-g  la 

small,  and  we  similarly  accept  since  the  risk  J?21 (1— ia  small. 

In  order  to  determine  the  sise  of  g  for  acceptance  of  H^,  the  whole 

interval  is  divided  into  three  subintervals  .*1  »  *2  j  *3  i 

C  g  g  1 

where  I^o^gsg),  I2(g<g<g)  and  Ij  (gjg^l).  Then  if  g  belongs  to  I2 
we  continue  to  make  observations,  but  if  g  belongs  to  I1  or  1^  we  stop 
and  take  action:  accept  if  in  1^  and  accept  if  in  ly  If  we  take 
n  observations,  the  a  posteriori  probability  of  is  given  by  Bayes 
Theorem  (Ref.  5): 

where  is  the  probability  of  having  n  observations  under  and 
the  probability  of  n  observations  under  Hj.  On  pp.  226-230  in  Vol.  17, 
Econometrics,  general  formulas  are  developed  for  g  and  £  and  also  for  a 
number  of  special  frequency  distributions.  In  all  cases,  however,  it  is 
very  difficult  to  compute  the  operating  characteristics  involved  in  the 
equations  for  g  and  £.  In  particular,  the  losses  ^2  and^-j  can  only 
be  guessed  at.  In  the  case  where  the  hypotheses  Kj  and  H2  do  not  differ 
much  from  each  other  the  approximations  developed  by  Wald  give  fairly 
close  results. 

f)  Critical  Comments  on  Utility  Theory  and  Principles  of  Choice. 

Utility  theory  (as  stated  before)  is  at  the  present  still  very 
general  in  character.  The  constructing  of  the  utility  function  U(d;<3)  is 
extremely  difficult  and  highly  subjective.  However,  even  though  some 
statisticians  assert  that  the  theory  is  doomed  from  a  practical  standpoint, 
our  feeling  is  that  it  has  given  many  new  insights  to  statistical  decision 
theory  and  offers  possibilities  for  great  usefulness  in  the  future.  The 
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first  obvious  difficulty  with  utility  theory  is  that  numerical  values  can 
very  seldom  be  assigned  to  the  consequences  of  decisions.  This  usually 
rests  upon  an  infinity  of  paired  comparisons.  Some  statisticians  have 
tried  to  avoid  this  difficulty  by  making  a  finite  or  a  relatively  few 
paired  comparisons.  Verification  then  is  based  on  these  as  well  as  on 
the  assumption  that  the  utility  function  exists  and  is  linear. 

A  number  of  authors  have  developed  axiom  systems  in  order  to 
have  both  a  plausible  set  of  consistent  requirements  based  on  idealised 
human  preferences  as  well  as  a  procedure  for  proving  the  validity  of 
the  utility  assignments.  A  second  difficulty  arises  as  a  result  of  these 
axiom  systems  since  the  preferences  or  paired  comparisons  almost  never 
satisfy  the  stated  axioms. 

The  principles  of  ohoice  discussed  earlier  were  Walds  ninimax 
principle,  Bayes  principle,  and  modifications  by  Hurwlcz  and  Savage. 
Serious  critioisms,  objections,  and  drawbacks  have  been  cited  by  many 
statisticians  to  all  of  these.  In  orlticism  of  Walds  minlmax  principle 
we  quote  from  Savage  (Ref.  12):  "There  is  a  general  principle  for  finding 
minlmax  actions  when  the  number  of  states  and  mixed  actions  is  finite, 
but  it  leads  in  general  to  very  extensive  computations  and  is  not 
applicable  at  all  when  either  of  these  numbers  is  infinite.  Devices 
for  solving  special  classes  of  minimax  problems  are  therefore  much  sought 
after,  and  even  now  many  of  the  most  oommonplaoe  situations  of  statistics 
lead  to  difficult  minimax  problems.  Few,  if  any,  new  minlmax  solutions 
of  immediate  practical  importance  have  yet  been  found."  A  second  serious 
objeotion  to  Walds  minimax  principle  (as  stated  before)  is  based  on 
extreme  pessimism,  that  is  that  we  assume  nature  to  be  in  the  worst 


-  25  - 


possible  state. 

The  Savage  modification,  minimising  the  maximum  regret,  has  been 
criticised  by  Chemoff  (Ref.  5)  as  follows: 

"Unfortunately,  the  minimax  regret  (risk)  criterion  has  several 
drawbacks.  First,  it  has  never  been  clearly  demonstrated  that  differences 
in  utility  do  in  fact  measure  what  one  may  call  regret  (risk).  In  other 
words,  it  is  not  clear  that  the  "regret"  of  going  from  a  state  of  utility 
5  to  a  state  of  utility  3  is  equivalent  in  some  sense  to  that  of  going 
from  a  state  of  utility  11  to  one  of  utility  9.  Secondly,  one  may  construct 
examples  where  an  arbitrarily  small  advantage  in  one  state  of  nature  out¬ 
weighs  a  considerable  advantage  in  another  state." 

Objections  to  the  principle  of  insufficient  reason  have  been 
many.  The  first  critioism  is  that  the  principle  is  extremely  vague  and 
may  lead  to  contradictory  results.  A  second  objection  is  that  the 
principle  is  not  strictly  applicable  for  a  person  who  has  had  any  ex¬ 
perience  with  the  problem  at  hand  since  utility  theory  depends  on  a  series 
of  preferences.  A  third  criticism,  much  like  the  first,  is  that  this 
criterion  is  highly  subjective.  For  example,  if  we  are  faced  with  a 
real  problem  in  decision  making,  we  oust  first  list  the  mutually  ex¬ 
clusive  states  of  nature.  The  objection  here  is  that  many  such  lists 
are  possible,  and  therefore  will  in  general  give  different  results. 

Finally,  the  fourth  and  last  principle  of  choice  discussed  was  the 
Hurwics  modification  of  Wald  minimax  principle  in  an  effort  to  make  it 
less  pessimistic.  The  major  objection  to  this  modification  is  that  it 
involves  the  selection  of  an  ^(considered  as  measuring  the  optimism  of 
the  statistician  flayer  Il7in  a  game  against  nature  flayer  l7  which 


i s  highly  subjective  and  vague.  A  criticism  which  applies  not  only  to 
the  Iiurvlcs  ck  modification  but  also  to  the  other  principles  of  choice 
discussed  is  that  they  depend  on  some  notion  of  complete  ignorance.  In 
reality,  however,  the  statistician  usually  is  not  in  complete  ignorance 
of  the  true  state  of  nature  but  has  some  idea  of  the  various 


possibilities. 
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CONCLUSION 

Since  most  methods  used  in  statistics  were  derived  upon  the 
basis  of  random  sampling,  it  is  essential  that  this  condition  be 
satisfied  in  order  that  the  results  obtained  by  valid.  In  particular, 
when  the  tests  involve  sequences  of  observations  ordered  with  respect 
to  time,  some  test  for  randomness  should  be  applied  before  methods  based 
on  randomness  are  used.  The  Runs  Test,  discussed  and  illustrated  in 
section  1b,  is  one  of  the  easiest  to  apply.  The  computations  are  single, 
the  test  is  nonparametric  (knowledge  of  underlying  frequency  function 
not  necessary),  and  can  be  used  for  both  small  (see  Table  I)  and  large 
samples.  The  Serial  Correlation  Test,  1c,  involves  laborious  computations' 
for  large  n  but  for  smaller  n  the  arithmetic  is  not  too  time  consuming. 

It  should  be  pointed  out  that  in  the  case  of  nonparametric  tests  there 
is  no  established  theory  to  determine  what  constitutes  a  "best"  test. 

The  construction  of  utility  functions  and  applications  of  decision 
theory  is  difficult  and  subjective.  However,  we  conclude  that: 

(1)  A  subjective  risk  function  or  decision  function  may  well 
be  an  optimum  one  if  the  individual  designing  it  has  knowledge  of  what 
is  important  and  what  is  not  important  to  the  purpose  of  the  test.  In 
the  particular  instance  of  a  risk  function  applied  to  a  reliability  test, 
the  Air  Force  (consumer)  should  have  the  prerogative  of  choosing  which 
parameters  of  a  risk  function  are  most  important  and  of  designing  the 
function  to  suit  these. 

(2)  While  decision  functions  in  general  are  difficult  to  mani¬ 
pulate  due  to  the  lack  of  a  common  denominator  for  different  con¬ 
sequences,  in  the  particular  case  of  a  decision  function  for  reliability 


test  purposes  it  may  be  found  to  be  less  difficult.  This  is  due  to  the 
feet  that  the  majority  of  aetions  involved  may  be  broken  down  to  but  two 
prime  factors,  time  and  money,  and  the  definition  of  the  relationship 
and  relative  importance  of  those  should  be  the  responsibility  of  the 
consumer. 

(3)  It  is  conceivable  that  a  decision  function  tailored  to 
reliability  tests  will  be  an  eventuality.  However,  before  this  eventuality 
may  be  realised,  more  knowledge  of  decision  functions  is  necessary.  At 
this  time  little  is  known  about  possible  applications  of  decision  functions, 
or  their  practicality  of  use.  This  area  first  came  into  prominence  about 
15  years  ago,  and  significant  work  was  accomplished.  However,  since  that 
time  little  more  has  been  accomplished  in  this  area  than  the  study  of  its 
more  obvious  fundamentals. 
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